Using historical statistics to predict future outcomes, wins and losses, and opportunities for improving performance has gained significant attention in professional sports.
This analysis aims to develop several models to predict a baseball team’s wins over a season based on team stats such as home runs, strikeouts, base hits, and more.
Our approach:
We will begin by examining the data for missing values or outliers and take the necessary measures to clean the data.
We will train and evaluate four different multivariate linear models that forecast seasonal wins using variants of the dataset split into training and evaluation sets.
Finally, we will choose the best model that balances accuracy and simplicity for predicting seasonal wins.
The baseball training dataset contains 2,276 observations of 17 variables detailing various teams’ performances from 1871 to 2006. The descriptions of each column are below.
Some initial observations:
Due to the relatively long period, we expect to see outliers and missing data as the league modified official game rules; these rule changes undoubtedly caused teams and players to change their tactics in response.
The number of single base hits is noticeably missing from the dataset, but we can derive this value by subtracting other hit types (doubles, triples, home runs) from total hits.
Lastly, other columns representing game number (out of 162), inning number (1-9), and matching opponent columns would be helpful for predictions.
The table below provides valuable descriptive statistics about the training data. These variables are all integers, and many have a minimum of zero.
One interesting piece of information is the min/max of the TARGET_WINS variable, where a value of zero suggests teams that did not win a single game in a given season. The maximum is 146, indicating no teams in the training dataset had a perfect 162-game season.
Also of note is the number of missing values from certain variables, which may be due to changes in official rules or tactics before the modern era. We’ll explore and handle these missing values in a later step.
Numerous extreme cases seem implausible; several teams have zero strikeouts by their pitchers over the season, and one achieved 20,000 strikeouts. We’ll also address these values in the coming steps.
| No | Variable | Stats / Values | Freqs (% of Valid) | Graph | Valid | Missing | ||||
|---|---|---|---|---|---|---|---|---|---|---|
| 1 | INDEX [integer] |
|
2276 distinct values | 2276 (100.0%) | 0 (0.0%) | |||||
| 2 | TARGET_WINS [integer] |
|
108 distinct values | 2276 (100.0%) | 0 (0.0%) | |||||
| 3 | TEAM_BATTING_H [integer] |
|
569 distinct values | 2276 (100.0%) | 0 (0.0%) | |||||
| 4 | TEAM_BATTING_2B [integer] |
|
240 distinct values | 2276 (100.0%) | 0 (0.0%) | |||||
| 5 | TEAM_BATTING_3B [integer] |
|
144 distinct values | 2276 (100.0%) | 0 (0.0%) | |||||
| 6 | TEAM_BATTING_HR [integer] |
|
243 distinct values | 2276 (100.0%) | 0 (0.0%) | |||||
| 7 | TEAM_BATTING_BB [integer] |
|
533 distinct values | 2276 (100.0%) | 0 (0.0%) | |||||
| 8 | TEAM_BATTING_SO [integer] |
|
822 distinct values | 2174 (95.5%) | 102 (4.5%) | |||||
| 9 | TEAM_BASERUN_SB [integer] |
|
348 distinct values | 2145 (94.2%) | 131 (5.8%) | |||||
| 10 | TEAM_BASERUN_CS [integer] |
|
128 distinct values | 1504 (66.1%) | 772 (33.9%) | |||||
| 11 | TEAM_BATTING_HBP [integer] |
|
55 distinct values | 191 (8.4%) | 2085 (91.6%) | |||||
| 12 | TEAM_PITCHING_H [integer] |
|
843 distinct values | 2276 (100.0%) | 0 (0.0%) | |||||
| 13 | TEAM_PITCHING_HR [integer] |
|
256 distinct values | 2276 (100.0%) | 0 (0.0%) | |||||
| 14 | TEAM_PITCHING_BB [integer] |
|
535 distinct values | 2276 (100.0%) | 0 (0.0%) | |||||
| 15 | TEAM_PITCHING_SO [integer] |
|
823 distinct values | 2174 (95.5%) | 102 (4.5%) | |||||
| 16 | TEAM_FIELDING_E [integer] |
|
549 distinct values | 2276 (100.0%) | 0 (0.0%) | |||||
| 17 | TEAM_FIELDING_DP [integer] |
|
144 distinct values | 1990 (87.4%) | 286 (12.6%) |
Generated by summarytools 1.0.1 (R version 4.2.2)
2023-02-25
Next, we’ll visually check for normality and outliers in the dependent and independent variables.
The density plots display near-normal distributions for most variables with some exceptions. Right-skewness is evident for PITCHING_H (hits allowed) and FIELDING_E (fielding errors), and bimodal distributions are observed in the density plots for BATTING_HR (home runs) and BATTING_SO (strikeouts), implying two distinct clusters of high-scoring and low-scoring teams.
Additionally, the boxplots indicate a large number of outliers exist outside of the interquartile ranges for many of the variables - all these effects will need to be carefully considered.
The function featurePlot() demonstrates the relationships between
each independent variable and the target TARGET_WINS.
Plotting the correlations between the target TARGET_WINS and the variables (excluding INDEX and TEAM_BATTING_HBP), we can see that very few variables correlate with the target. Variables with correlations close to zero are unlikely to offer significant insights into the factors contibuting to a team’s victories.
| Coefficient | |
|---|---|
| TARGET_WINS | 1.0000000 |
| TEAM_BATTING_H | 0.4699467 |
| TEAM_BATTING_2B | 0.3129840 |
| TEAM_BATTING_3B | -0.1243459 |
| TEAM_BATTING_HR | 0.4224168 |
| TEAM_BATTING_BB | 0.4686879 |
| TEAM_BATTING_SO | -0.2288927 |
| TEAM_BASERUN_SB | 0.0148364 |
| TEAM_BASERUN_CS | -0.1787560 |
| TEAM_BATTING_HBP | 0.0735042 |
| TEAM_PITCHING_H | 0.4712343 |
| TEAM_PITCHING_HR | 0.4224668 |
| TEAM_PITCHING_BB | 0.4683988 |
| TEAM_PITCHING_SO | -0.2293648 |
| TEAM_FIELDING_E | -0.3866880 |
| TEAM_FIELDING_DP | -0.1958660 |
To avoid multicollinearity, we should exclude some variables from the model that have high degrees of correlation with each other. By comparing offensive with defensive stats (variables starting with BATTING or BASERUN), we can see potential cases of multicollinearity. We might interpret this as some teams being exceptional at both hitting (offense) and fielding (defense).
Further, a typical team’s number of TEAM_BATTING_HR (home runs) and TEAM_PITCHING_HR (allowed home runs) has a correlation of nearly 1.0. This is an unexpected correlation but can be explained by noticing most games are decided by a difference of one or two runs. Any final models should probably exclude one of these two home run variables.
Alternatively, the correlation between a team’s BATTING_H (hits) and PITCHING_H (hits allowed) is around 0.3 which seems reasonable.
Some other strong correlations are less obvious such as TEAM_FIELDING_E (errors) being strongly negatively correlated with TEAM_BATTING_BB (walks by batters) and TEAM_BATTING_HR (home runs).
Digging a little deeper we can see that TEAM_FIELDING_E (errors) has a strong negative Pearson correlation coefficient of r round(cor(trainDf$TEAM_FIELDING_E, trainDf$TEAM_BATTING_BB),3) with TEAM_BATTING_BB (walks by batters), and a strong positive correlation of r round(cor(trainDf$TEAM_FIELDING_E, trainDf$TEAM_PITCHING_H),3) with TEAM_PITCHING_H (pitching hits allowed). Altogether, this seems to support an overall correlation of strong offensive stats with strong defensive stats (and vice versa.)
Let’s take a closer look at the missing data using the plot below, highlighting six variables with various percentages of missing values:
Most notably, TEAM_BATTING_HBP (batter hit by pitch) is missing values in 91% of the cases; we’ll remove this variable from the dataset because there is not enough information to estimate sensible values.
The variable TEAM_BASERUN_CS (caught stealing) is also missing data in 34% of the cases; instead of dropping this variable and losing the information present, we’ll use imputation methods to fill in values we might reasonably expect.
The remaining variables are missing data in fewer than 25% of the cases, so we’ll impute expected values here too.
Additionally, we’ll drop the INDEX variable as it is just an identification label for each case.
To handle the cases with missing values, we’ll create a new variable na_flag to label if any variables were missing for each case, and use the MICE (Multiple Imputation by Chained Equations) library to impute values for our missing data.
One of the assumptions of MICE is that the missing values in our dataset occur randomly, and be estimated by using the existing values in the rest of the dataset. The MICE algorithm performs several iterations over the data and generates data to complete the missing values.
One of the most common algorithms used by MICE is Predictive Mean Matching (PMM), which imputes missing values based on the means of the existing values.
Here are the new density plots of our five variables after imputation with MICE. The blue plots represent the original values with missing data, and the red plots indicate multiple passes of the MICE algorithm with imputed values.
After imputation, the distribution of TEAM_BATTING_SO
becomes roughly more normally distributed, so it may be beneficial for
the model. However, the remaining variables may need to be handled via
alternative methods.
After exploring outliers using methods based on standard deviation and IQR, we decided to try another approach to preserve as much information for our models as possible.
We used historical data from the Lahman’s Baseball Database to identify outliers. Early baseball teams played fewer than 20 games per season, and the calculation used in our project dataset to normalize the statistics to a 162-game season tended to create outliers.
We used the min and max from the modern era (post-1900) as a filter to alleviate the issues created by outlier values. We compared each statistic listed in the column names to the min and max for the same statistic in the modern era, and we dropped the row if the values were below the min or above the max.
| No | Variable | Stats / Values | Freqs (% of Valid) | Graph | Valid | Missing | ||||
|---|---|---|---|---|---|---|---|---|---|---|
| 1 | TARGET_WINS [integer] |
|
80 distinct values | 1948 (100.0%) | 0 (0.0%) | |||||
| 2 | TEAM_BATTING_H [integer] |
|
480 distinct values | 1948 (100.0%) | 0 (0.0%) | |||||
| 3 | TEAM_BATTING_2B [integer] |
|
225 distinct values | 1948 (100.0%) | 0 (0.0%) | |||||
| 4 | TEAM_BATTING_3B [integer] |
|
116 distinct values | 1948 (100.0%) | 0 (0.0%) | |||||
| 5 | TEAM_BATTING_HR [integer] |
|
240 distinct values | 1948 (100.0%) | 0 (0.0%) | |||||
| 6 | TEAM_BATTING_BB [integer] |
|
405 distinct values | 1948 (100.0%) | 0 (0.0%) | |||||
| 7 | TEAM_BATTING_SO [integer] |
|
747 distinct values | 1948 (100.0%) | 0 (0.0%) | |||||
| 8 | TEAM_BASERUN_SB [integer] |
|
271 distinct values | 1948 (100.0%) | 0 (0.0%) | |||||
| 9 | TEAM_BASERUN_CS [integer] |
|
126 distinct values | 1948 (100.0%) | 0 (0.0%) | |||||
| 10 | TEAM_PITCHING_H [integer] |
|
599 distinct values | 1948 (100.0%) | 0 (0.0%) | |||||
| 11 | TEAM_PITCHING_HR [integer] |
|
241 distinct values | 1948 (100.0%) | 0 (0.0%) | |||||
| 12 | TEAM_PITCHING_BB [integer] |
|
432 distinct values | 1948 (100.0%) | 0 (0.0%) | |||||
| 13 | TEAM_PITCHING_SO [integer] |
|
746 distinct values | 1948 (100.0%) | 0 (0.0%) | |||||
| 14 | TEAM_FIELDING_E [integer] |
|
326 distinct values | 1948 (100.0%) | 0 (0.0%) | |||||
| 15 | TEAM_FIELDING_DP [integer] |
|
138 distinct values | 1948 (100.0%) | 0 (0.0%) |
Generated by summarytools 1.0.1 (R version 4.2.2)
2023-02-25
We’ll also derive a new column TEAM_BATTING_1B (single base hits) by subtracting doubles, triples, and home runs from the total number of hits per case.
We should also try to transform some variables to fit a more normal
distribution, including TEAM_BATTING_HR,
TEAM_BATTING_SO, TEAM_PITCHING_HR,
TEAM_PITCHING_H, TEAM_PITCHING_BB,
TEAM_PITCHING_SO, and TEAM_FIELDING_E.
We apply a square-root transformation to these variables and remove the skewness by reducing the low-density ranges. The plots below compare the original distributions of non-normal variables and transformed ones.
Our current dataset is free of missing data values, the irrelevant INDEX and the TEAM_BATTING_HBP variables, and the outliers. As shown in the table below, no missing data values exist anymore, and we can analyze how the summary statistics may have altered with the imputed data.
| No | Variable | Stats / Values | Freqs (% of Valid) | Graph | Valid | Missing | ||||
|---|---|---|---|---|---|---|---|---|---|---|
| 1 | TARGET_WINS [integer] |
|
108 distinct values | 2276 (100.0%) | 0 (0.0%) | |||||
| 2 | TEAM_BATTING_H [integer] |
|
569 distinct values | 2276 (100.0%) | 0 (0.0%) | |||||
| 3 | TEAM_BATTING_2B [integer] |
|
240 distinct values | 2276 (100.0%) | 0 (0.0%) | |||||
| 4 | TEAM_BATTING_3B [integer] |
|
144 distinct values | 2276 (100.0%) | 0 (0.0%) | |||||
| 5 | TEAM_BATTING_HR [integer] |
|
243 distinct values | 2276 (100.0%) | 0 (0.0%) | |||||
| 6 | TEAM_BATTING_BB [integer] |
|
533 distinct values | 2276 (100.0%) | 0 (0.0%) | |||||
| 7 | TEAM_BATTING_SO [integer] |
|
822 distinct values | 2276 (100.0%) | 0 (0.0%) | |||||
| 8 | TEAM_BASERUN_SB [integer] |
|
348 distinct values | 2276 (100.0%) | 0 (0.0%) | |||||
| 9 | TEAM_BASERUN_CS [integer] |
|
128 distinct values | 2276 (100.0%) | 0 (0.0%) | |||||
| 10 | TEAM_PITCHING_H [integer] |
|
843 distinct values | 2276 (100.0%) | 0 (0.0%) | |||||
| 11 | TEAM_PITCHING_HR [integer] |
|
256 distinct values | 2276 (100.0%) | 0 (0.0%) | |||||
| 12 | TEAM_PITCHING_BB [integer] |
|
535 distinct values | 2276 (100.0%) | 0 (0.0%) | |||||
| 13 | TEAM_PITCHING_SO [integer] |
|
823 distinct values | 2276 (100.0%) | 0 (0.0%) | |||||
| 14 | TEAM_FIELDING_E [integer] |
|
549 distinct values | 2276 (100.0%) | 0 (0.0%) | |||||
| 15 | TEAM_FIELDING_DP [integer] |
|
144 distinct values | 2276 (100.0%) | 0 (0.0%) | |||||
| 16 | batting_hr_sqrt [numeric] |
|
243 distinct values | 2276 (100.0%) | 0 (0.0%) | |||||
| 17 | batting_so_sqrt [numeric] |
|
822 distinct values | 2276 (100.0%) | 0 (0.0%) | |||||
| 18 | baserun_cs_sqrt [numeric] |
|
128 distinct values | 2276 (100.0%) | 0 (0.0%) | |||||
| 19 | pitching_hr_sqrt [numeric] |
|
256 distinct values | 2276 (100.0%) | 0 (0.0%) |
Generated by summarytools 1.0.1 (R version 4.2.2)
2023-02-25
Having thoroughly explored our dataset and completed the data cleaning process, we can initiate the construction of our multivariate linear regression models.
We’ll build our models using four variants of our training dataset:
The original training dataset (trainDf),
The training dataset with missing data variables imputed using MICE (cleanDf),
The training dataset with outliers dropped, using stepwise selection (droppedDf), and
The dataset with transforms of non-normal variables, using stepwise selection (transformedDf).
For each model, we will split the dataset into training and testing sets (80% training, 20% testing), evaluating each model’s performance on the test set.
For the first multivariate linear regression model, we will select
all the variables from the original un-cleaned dataset except for
INDEX (just a row index) and TEAM_BATTING_HBP
(missing data in 91% of cases). We may use this model as a base model
for comparison.
| Â | TARGET_WINS | |||
|---|---|---|---|---|
| Predictors | Estimates | std. Error | Statistic | p |
| (Intercept) | 57.91 | 6.64 | 8.72 | <0.0001 |
| TEAM BATTING H | 0.02 | 0.02 | 0.79 | 0.4318 |
| TEAM BATTING 2B | -0.07 | 0.01 | -7.52 | <0.0001 |
| TEAM BATTING 3B | 0.16 | 0.02 | 7.28 | <0.0001 |
| TEAM BATTING HR | 0.07 | 0.09 | 0.87 | 0.3866 |
| TEAM BATTING BB | 0.04 | 0.05 | 0.94 | 0.3463 |
| TEAM BATTING SO | 0.02 | 0.02 | 0.78 | 0.4368 |
| TEAM BASERUN SB | 0.04 | 0.01 | 4.13 | <0.0001 |
| TEAM BASERUN CS | 0.05 | 0.02 | 2.86 | 0.0043 |
| TEAM PITCHING H | 0.02 | 0.02 | 1.04 | 0.3003 |
| TEAM PITCHING HR | 0.02 | 0.08 | 0.28 | 0.7794 |
| TEAM PITCHING BB | -0.00 | 0.04 | -0.09 | 0.9255 |
| TEAM PITCHING SO | -0.04 | 0.02 | -1.70 | 0.0892 |
| TEAM FIELDING E | -0.16 | 0.01 | -15.67 | <0.0001 |
| TEAM FIELDING DP | -0.11 | 0.01 | -8.59 | <0.0001 |
| Observations | 1486 | |||
| R2 / R2 adjusted | 0.439 / 0.433 | |||
The F-statistic is 82.1, the adjusted R-squared is 0.433, and out of the 15 variables, 6 have statistically significant p-values.
The adjusted R2 indicates that only 43% of the variance in the response variable can be explained by the predictor variables. The F-statistic is low and the model’s p-value is not statistically significant. We should probably look at other models for better performance.
We evaluate the linear modeling assumptions using standard diagnostic plots, the Breusch–Pagan Test for Heteroscedasticity and Variance Inflation Factor (VIF) to assess colinearity.
The initial review of the diagnostic plots for this model shows some deviation from linear modeling assumptions. The Q-Q plot shows some deviations from the normal distribution at the ends. The residuals-fitted and standardized residuals-fitted plots show a curve in the main cluster, which indicates that we do not have constant variance.
Our second model will be based on the cleaned dataset, excluding
variables that contain missing values. We chose the variables
TEAM_PITCHING_BB, TEAM_BATTING_2B,
TEAM_BATTING_3B, TEAM_FIELDING_E,
TEAM_PITCHING_H, TEAM_BATTING_HR,
TEAM_BATTING_H based on our understanding of the data.
| Â | TARGET_WINS | |||
|---|---|---|---|---|
| Predictors | Estimates | std. Error | Statistic | p |
| (Intercept) | 7.27 | 3.28 | 2.22 | 0.0266 |
| TEAM PITCHING BB | 0.01 | 0.00 | 5.29 | <0.0001 |
| TEAM BATTING 2B | -0.03 | 0.01 | -2.85 | 0.0044 |
| TEAM BATTING 3B | 0.10 | 0.02 | 6.07 | <0.0001 |
| TEAM FIELDING E | -0.02 | 0.00 | -7.36 | <0.0001 |
| TEAM PITCHING H | -0.00 | 0.00 | -4.03 | 0.0001 |
| TEAM BATTING HR | 0.04 | 0.01 | 4.85 | <0.0001 |
| TEAM BATTING H | 0.05 | 0.00 | 15.22 | <0.0001 |
| Observations | 2276 | |||
| R2 / R2 adjusted | 0.270 / 0.268 | |||
The F-statistic is 119.9, the adjusted R-squared is 0.268, and all 7 variables have statistically significant p-values.
The adjusted R2 indicates that only 26% of the variance in the response variable can be explained by the predictor variables, which is less than in the original model.
The F-statistic is high, the p-value of the model is close to zero, and the model’s diagnostic checks are satisfactory. We would dismiss the null hypothesis, which assumes no correlation between the explanatory and response variables.
The initial review of the diagnostic plots for this model shows some deviation from linear modeling assumptions. The Q-Q plot shows the normal distribution, but the residuals-fitted and standardzied residuals-fitted plots don’t have constant variance for the fitted values below 60 and above 95.
The third model uses the cleaned dataset with outliers omitted, and
we dropped TEAM_BATTING_H due to colinearity with
TEAM_BATTING_1B, TEAM_BATTING_2B,
TEAM_BATTING_3B and TEAM_BATTING_HR.
We use stepwise variable selection based on their AIC score to identify the optimal set of features. The resulting model has significant p-values for the model and all predictor variables.
| Â | TARGET_WINS | |||
|---|---|---|---|---|
| Predictors | Estimates | std. Error | Statistic | p |
| (Intercept) | 62.72 | 5.60 | 11.21 | <0.0001 |
| TEAM BATTING 2B | -0.04 | 0.01 | -4.96 | <0.0001 |
| TEAM BATTING 3B | 0.18 | 0.02 | 9.68 | <0.0001 |
| TEAM BATTING HR | 0.09 | 0.01 | 10.05 | <0.0001 |
| TEAM BATTING BB | 0.15 | 0.02 | 8.25 | <0.0001 |
| TEAM BATTING SO | -0.04 | 0.01 | -5.67 | <0.0001 |
| TEAM BASERUN SB | 0.09 | 0.01 | 15.94 | <0.0001 |
| TEAM BASERUN CS | -0.06 | 0.01 | -5.84 | <0.0001 |
| TEAM PITCHING H | 0.03 | 0.00 | 7.09 | <0.0001 |
| TEAM PITCHING BB | -0.11 | 0.02 | -6.61 | <0.0001 |
| TEAM PITCHING SO | 0.02 | 0.01 | 2.49 | 0.0128 |
| TEAM FIELDING E | -0.12 | 0.01 | -20.45 | <0.0001 |
| TEAM FIELDING DP | -0.12 | 0.01 | -9.81 | <0.0001 |
| Observations | 1948 | |||
| R2 / R2 adjusted | 0.423 / 0.419 | |||
| AIC | 14671.497 | |||
The F-statistic is 117.3, the adjusted R-squared 0.419, and all 12 variables have statistically significant p-values.
The adjusted R2 indicates that 42% of the variance in the response variable can be explained by the predictor variables. The F-statistic is high, and the p-value of the model is close to zero, so we would dismiss the null hypothesis, which assumes that there is no correlation between the explanatory and response variables.
The initial review of the diagnostic plots for this model shows some deviation from linear modeling assumptions at the boundaries of the prediction range, but the overall QQ plot is almost a flat line; the residuals are flat and exhibit homoscedasticity of variance in the 65 - 95 fitted value range.
The Breusch–Pagan Test for Heteroscedasticity assumes the following Null and Alternate hypothesis.
For this model iteration, we reject the null hypothesis and conclude that this model violates the homoscedasticity function.
studentized Breusch-Pagan test
data: lm3step BP = 68.868, df = 12, p-value = 5.213e-10
The Variance Inflation Factor (VIF) calculation detects collinearity
between the TEAM_PITCHING_BB and
TEAM_BATTING_BB variables. Both variables have a VIF score
over 5 and are 0.93 correlated. Additionally, variables
TEAM_PITCHING_SO, TEAM_BATTING_SO,
TEAM_PITCHING_H and TEAM_BATTING_2B have VIF
over 5 as well.
For the refined model we will drop TEAM_PITCHING_BB,
TEAM_BATTING_BB, TEAM_PITCHING_SO
TEAM_BATTING_SO TEAM_PITCHING_H and
TEAM_BATTING_2B.
| Â | TARGET_WINS | |||
|---|---|---|---|---|
| Predictors | Estimates | std. Error | Statistic | p |
| (Intercept) | 107.66 | 3.24 | 33.21 | <0.0001 |
| TEAM BATTING 3B | 0.21 | 0.02 | 12.17 | <0.0001 |
| TEAM BATTING HR | 0.14 | 0.01 | 19.48 | <0.0001 |
| TEAM BATTING SO | -0.03 | 0.00 | -16.74 | <0.0001 |
| TEAM BASERUN SB | 0.10 | 0.01 | 18.15 | <0.0001 |
| TEAM BASERUN CS | -0.07 | 0.01 | -5.74 | <0.0001 |
| TEAM FIELDING E | -0.13 | 0.01 | -21.39 | <0.0001 |
| TEAM FIELDING DP | -0.09 | 0.01 | -7.60 | <0.0001 |
| Observations | 1948 | |||
| R2 / R2 adjusted | 0.374 / 0.372 | |||
| AIC | 14817.770 | |||
This model has a lower adjusted R2 of 0.37, but with a high F-statistic of 163.9 and statistically significant p-values, it should align better with the linear regression assumptions.
The diagnostic plots for the new model appear to be more closely with the linear regression assumptions. We again reject the null hypothesis and conclude that this model violates the homoscedasticity function.
The Variance Inflation Factor (VIF) calculation does not detect collinearity across the remaining variables. Overall, however, we will reject Model 3 because the residuals are not homoscedastic.
studentized Breusch-Pagan test
data: lm3step.final BP = 45.198, df = 7, p-value = 1.252e-07
| Â | model 3 | model 3 (StepAIC) | model 3 (Final) | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Predictors | Estimates | std. Error | Statistic | p | Estimates | std. Error | Statistic | p | Estimates | std. Error | Statistic | p |
| (Intercept) | 62.92 | 5.81 | 10.83 | <0.0001 | 62.72 | 5.60 | 11.21 | <0.0001 | 107.66 | 3.24 | 33.21 | <0.0001 |
| TEAM BATTING 2B | -0.05 | 0.02 | -2.53 | 0.0116 | -0.04 | 0.01 | -4.96 | <0.0001 | ||||
| TEAM BATTING 3B | 0.17 | 0.02 | 7.17 | <0.0001 | 0.18 | 0.02 | 9.68 | <0.0001 | 0.21 | 0.02 | 12.17 | <0.0001 |
| TEAM BATTING HR | 0.12 | 0.09 | 1.38 | 0.1668 | 0.09 | 0.01 | 10.05 | <0.0001 | 0.14 | 0.01 | 19.48 | <0.0001 |
| TEAM BATTING BB | 0.15 | 0.05 | 3.05 | 0.0023 | 0.15 | 0.02 | 8.25 | <0.0001 | ||||
| TEAM BATTING SO | -0.04 | 0.01 | -5.50 | <0.0001 | -0.04 | 0.01 | -5.67 | <0.0001 | -0.03 | 0.00 | -16.74 | <0.0001 |
| TEAM BASERUN SB | 0.09 | 0.01 | 15.51 | <0.0001 | 0.09 | 0.01 | 15.94 | <0.0001 | 0.10 | 0.01 | 18.15 | <0.0001 |
| TEAM BASERUN CS | -0.06 | 0.01 | -5.65 | <0.0001 | -0.06 | 0.01 | -5.84 | <0.0001 | -0.07 | 0.01 | -5.74 | <0.0001 |
| TEAM PITCHING H | 0.03 | 0.02 | 1.84 | 0.0656 | 0.03 | 0.00 | 7.09 | <0.0001 | ||||
| TEAM PITCHING HR | -0.03 | 0.08 | -0.38 | 0.7067 | ||||||||
| TEAM PITCHING BB | -0.11 | 0.05 | -2.37 | 0.0177 | -0.11 | 0.02 | -6.61 | <0.0001 | ||||
| TEAM PITCHING SO | 0.02 | 0.01 | 2.47 | 0.0137 | 0.02 | 0.01 | 2.49 | 0.0128 | ||||
| TEAM FIELDING E | -0.12 | 0.01 | -20.13 | <0.0001 | -0.12 | 0.01 | -20.45 | <0.0001 | -0.13 | 0.01 | -21.39 | <0.0001 |
| TEAM FIELDING DP | -0.12 | 0.01 | -9.74 | <0.0001 | -0.12 | 0.01 | -9.81 | <0.0001 | -0.09 | 0.01 | -7.60 | <0.0001 |
| TEAM BATTING 1B | -0.00 | 0.02 | -0.14 | 0.8852 | ||||||||
| Observations | 1948 | 1948 | 1948 | |||||||||
| R2 / R2 adjusted | 0.423 / 0.418 | 0.423 / 0.419 | 0.374 / 0.372 | |||||||||
| AIC | 14675.322 | 14671.497 | 14817.770 | |||||||||
The fourth model uses the dataset with missing values imputed, outliers removed, and variables square-root transformed for more normal distributions. Stepwise AIC selection is applied to select the optimal combination of variables. The resulting model has significant p-values for the model and all predictor variables.
| Â | TARGET_WINS | |||
|---|---|---|---|---|
| Predictors | Estimates | std. Error | Statistic | p |
| (Intercept) | 37.08 | 5.62 | 6.60 | <0.0001 |
| TEAM BATTING H | 0.05 | 0.00 | 14.92 | <0.0001 |
| TEAM BATTING 2B | -0.02 | 0.01 | -2.67 | 0.0077 |
| TEAM BATTING BB | 0.01 | 0.00 | 3.86 | 0.0001 |
| TEAM BASERUN SB | 0.05 | 0.00 | 12.77 | <0.0001 |
| TEAM PITCHING SO | 0.00 | 0.00 | 3.43 | 0.0006 |
| TEAM FIELDING E | -0.04 | 0.00 | -17.46 | <0.0001 |
| TEAM FIELDING DP | -0.09 | 0.01 | -7.56 | <0.0001 |
| batting hr sqrt | 1.22 | 0.16 | 7.73 | <0.0001 |
| batting so sqrt | -0.82 | 0.11 | -7.30 | <0.0001 |
| baserun cs sqrt | -0.44 | 0.16 | -2.70 | 0.0069 |
| Observations | 2276 | |||
| R2 / R2 adjusted | 0.361 / 0.358 | |||
| AIC | 18012.287 | |||
The F-statistic is 98.67, the adjusted R-squared 0.358, and the p-value of the model is close to zero, so we would dismiss the null hypothesis, which assumes that there is no correlation between the explanatory and response variables.
The review of the diagnostic plots shows that the overall QQ plot is almost a flat line; the residuals are flat and exhibit homoscedasticity of variance in the 60 - 100 fitted value range.
The table presents an analysis of the performance metrics for each model on the training dataset. The results suggest that the model’s performance slightly improved after incorporating transformations and selecting significant parameters.
The $R^2$, residual standard error, and F-statistics for each model are below. The non-cleaned, non-imputed raw training data had the best-fitting statistics. Though Model 1 doesn’t meet assumption requirements. Model 3 looks like the best fit.
The F-statistic for models 2-3 was large, with significant p-values. During the exploratory data analysis, some of the predictor variables had correlations with each other, as expected in baseball. However, this high correlation between variables could result in multicollinearity, causing unstable regression fits. To mitigate this issue, we employed the Variable Inflation Factor (VIF) method in the previous section to identify better models by selecting variables with VIF values less than 5.
The residual plots from part 3 showed that models 1-3 almost follow the normal distribution and almost constant variance.
| r | mse | adjusted.r |
|---|---|---|
| 0.44 | 9.56 | 0.43 |
| 0.27 | 13.48 | 0.27 |
| 0.42 | 10.41 | 0.42 |
| 0.36 | 12.62 | 0.36 |
Next, we apply the models to the evaluation dataset to make predictions. However, to ensure that Models 2-4 work properly, we fill in the missing values in the evaluation data set using the same MICE imputation method, drop the INDEX and TEAM_BATTING_HBP variables, remove outliers for Model 3, and make transformations for Model 4.
This table presents the predicted values of TARGET_WINS for each model. The first thing to notice is that the first model predicts a negative number of wins, which is unrealistic. The third and fourth AIC-generated models (no outliers and transformed values) have similar outcomes, with both performing well. The second model also produced similar results but had an issue with model assumptions.
| lm1 | lm2 | aic3 | aic4 |
|---|---|---|---|
| 62.57 | 68.58 | 60.12 | 62.85 |
| 68.91 | 70.21 | 67.00 | 66.03 |
| 73.68 | 77.35 | 71.92 | 75.44 |
| 81.89 | 83.61 | 84.33 | 87.27 |
| 20.39 | 66.44 | 74.53 | 57.00 |
| 32.64 | 67.44 | 65.74 | 75.28 |
| 48.67 | 74.02 | 68.01 | 83.20 |
| 58.69 | 72.52 | 59.07 | 73.87 |
| 67.81 | 72.08 | 71.37 | 67.30 |
| 69.81 | 75.86 | 69.24 | 73.89 |
We can also see when plotting the predictions that there doesn’t seem to be much difference between the models, aside from the unrealistic outliers generated by Model 1, and the slightly tighter range in wins in Model 4.
We don’t generally place much faith in any of the models for their predictive power:
Model 1 produced the highest \(R^2\), but was affected by missing data leading to negative win predictions.
Model 3 produced lower \(R^2\) than Model 1 but didn’t produce negative results.
Models 2 and 4 demonstrated a weak overall fit, as evidenced by their lower \(R^2\) scores.
None of the models displayed satisfactory performance when assessed for linearity or homogeneity of variance.Â
According to the analysis, Models 1 and 3 performed slightly better than Models 2 and 4, and would be the preferred linear models based solely on their adjusted \(R^2\). However, Model 3 is more statistically significant than the others and uses fewer unnecessary variables for making predictions while maintaining a similar level of adjusted \(R^2\).
Overall, Model 3 would be the recommended choice based on these factors and its better handling of multicollinearity (with fewer instances of sign-flipping in coefficients.)
Despite significant efforts to compensate for poor data quality, the resulting models are poor predictors of win totals.
Moreover, the poor data quality is inconsistent with the overall state of baseball statistics. When it comes to the major sports, baseball has the most mature statistics available. Therefore finding better data is the best course of action for developing a better predictive model.
We were able to locate a cleaner version of the same data set provided in the class. The Lahman’s Baseball Database includes the same variables as the sample database with fewer errors and additional reference data that would allow us to connect the database to other sources.
https://www.seanlahman.com/baseball-archive/statistics/
A significant advantage of Lahman’s data set over the data set provided in class is that it includes information about the year and the team. This data is valuable when considering how baseball has changed over the years. The modern era in baseball is often delineated by the turn of the century. However, when looking at the past 120 years of baseball history, it is easy to pinpoint rule changes, evolutions in playing strategy, and league structure that have fundamentally impacted the game.
When comparing statistics across time, it is common to use many of the breakdowns below to add context to the analysis:
Surveying these periods would suggest that a more granular model has the potential to perform better.
Although we could have chosen any number of the time periods above, exploring the statistical outliers highlights that many of these values correspond to the pre-1969 period. This delineation has some historical support.
As Jayson Stark of ESPN argues in this article (https://www.espn.com/mlb/columns/story?columnist=stark_jayson&id=2471349) In 1969 the MLB underwent several rule changes and changes to the league structure that impacted win totals and team statistics. 1969 was the first year of division play and the expanded postseason. The Pitcher’s Mound was lowered five inches. The Strike zone shrinks. Five-person rotations kicking in. The save was invented. And more expansion to the unbalanced schedules.
Thus using 1900 as the beginning of the modern era and 1969 as an additional breakpoint, the dataset can be divided into three segments. The density profiles for the predictor variables approach a normal distribution when grouped by the three segments we identified. To support data exploration, we added a era_cat field to the data set.
In general the Lahman’s data set contains fewer data gaps and the variables are more consistently distributed. There are some missing values in the data set including the Caught Stealing (CS/TEAM_BASERUN_CS) variable is missing 27.9%; the Batters Hit by Pitch (HBP/TEAM_BATTING_HBP) variable is missing 38.8%; and the Sacrifice Flies (SF) variable is missing 51.6% of the values.
Most of the variables in the data set show some level of skewness, with the following variables having a Kurtosis measure of greater than 3, TEAM_BATTING_H, TEAM_BASERUN_SB, TEAM_BASERUN_CS, TEAM_PITCHING_H, and TEAM_FIELDING_E
The Lahman data set contains several variables with bimodal distributions, including, TEAM_BATTING_HR, TEAM_BATTING_SO, TEAM_BATTING_HR, and TEAM_PITCHING_SO.
| No | Variable | Stats / Values | Freqs (% of Valid) | Graph | Valid | Missing | |||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | yearID [numeric] |
|
151 distinct values | 2985 (100.0%) | 0 (0.0%) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 2 | lgID [character] |
|
|
2935 (98.3%) | 50 (1.7%) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 3 | teamID [character] |
|
|
2985 (100.0%) | 0 (0.0%) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 4 | franchID [character] |
|
|
2985 (100.0%) | 0 (0.0%) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 5 | divID [character] |
|
|
2985 (100.0%) | 0 (0.0%) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 6 | Rank [numeric] |
|
13 distinct values | 2985 (100.0%) | 0 (0.0%) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 7 | G [numeric] |
|
123 distinct values | 2985 (100.0%) | 0 (0.0%) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 8 | W [numeric] |
|
113 distinct values | 2985 (100.0%) | 0 (0.0%) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 9 | L [numeric] |
|
114 distinct values | 2985 (100.0%) | 0 (0.0%) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 10 | R [numeric] |
|
640 distinct values | 2985 (100.0%) | 0 (0.0%) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 11 | AB [numeric] |
|
1137 distinct values | 2985 (100.0%) | 0 (0.0%) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 12 | H [numeric] |
|
758 distinct values | 2985 (100.0%) | 0 (0.0%) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 13 | X2B [numeric] |
|
317 distinct values | 2985 (100.0%) | 0 (0.0%) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 14 | X3B [numeric] |
|
126 distinct values | 2985 (100.0%) | 0 (0.0%) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 15 | HR [numeric] |
|
260 distinct values | 2985 (100.0%) | 0 (0.0%) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 16 | BB [numeric] |
|
586 distinct values | 2984 (100.0%) | 1 (0.0%) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 17 | SO [numeric] |
|
1117 distinct values | 2969 (99.5%) | 16 (0.5%) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 18 | SB [numeric] |
|
324 distinct values | 2859 (95.8%) | 126 (4.2%) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 19 | CS [numeric] |
|
137 distinct values | 2153 (72.1%) | 832 (27.9%) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 20 | HBP [numeric] |
|
101 distinct values | 1827 (61.2%) | 1158 (38.8%) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 21 | SF [numeric] |
|
66 distinct values | 1444 (48.4%) | 1541 (51.6%) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 22 | RA [numeric] |
|
623 distinct values | 2985 (100.0%) | 0 (0.0%) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 23 | ER [numeric] |
|
656 distinct values | 2985 (100.0%) | 0 (0.0%) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 24 | ERA [numeric] |
|
|
2985 (100.0%) | 0 (0.0%) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 25 | CG [numeric] |
|
147 distinct values | 2985 (100.0%) | 0 (0.0%) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 26 | SHO [numeric] |
|
32 distinct values | 2985 (100.0%) | 0 (0.0%) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 27 | SV [numeric] |
|
66 distinct values | 2985 (100.0%) | 0 (0.0%) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 28 | IPouts [numeric] |
|
862 distinct values | 2985 (100.0%) | 0 (0.0%) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 29 | HA [numeric] |
|
770 distinct values | 2985 (100.0%) | 0 (0.0%) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 30 | HRA [numeric] |
|
247 distinct values | 2985 (100.0%) | 0 (0.0%) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 31 | BBA [numeric] |
|
577 distinct values | 2985 (100.0%) | 0 (0.0%) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 32 | SOA [numeric] |
|
1148 distinct values | 2985 (100.0%) | 0 (0.0%) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 33 | E [numeric] |
|
474 distinct values | 2985 (100.0%) | 0 (0.0%) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 34 | DP [numeric] |
|
199 distinct values | 2985 (100.0%) | 0 (0.0%) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 35 | FP [numeric] | 1 distinct value |
|
2985 (100.0%) | 0 (0.0%) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 36 | name [character] |
|
|
2985 (100.0%) | 0 (0.0%) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 37 | gFactor [numeric] |
|
14 distinct values | 2985 (100.0%) | 0 (0.0%) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 38 | TARGET_WINS [numeric] |
|
111 distinct values | 2985 (100.0%) | 0 (0.0%) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 39 | TEAM_BATTING_H [numeric] |
|
604 distinct values | 2985 (100.0%) | 0 (0.0%) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 40 | TEAM_BATTING_2B [numeric] |
|
248 distinct values | 2985 (100.0%) | 0 (0.0%) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 41 | TEAM_BATTING_3B [numeric] |
|
149 distinct values | 2985 (100.0%) | 0 (0.0%) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 42 | TEAM_BATTING_HR [numeric] |
|
263 distinct values | 2985 (100.0%) | 0 (0.0%) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 43 | TEAM_BATTING_BB [numeric] |
|
563 distinct values | 2984 (100.0%) | 1 (0.0%) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 44 | TEAM_BATTING_SO [numeric] |
|
1077 distinct values | 2969 (99.5%) | 16 (0.5%) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 45 | TEAM_BASERUN_SB [numeric] |
|
362 distinct values | 2859 (95.8%) | 126 (4.2%) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 46 | TEAM_BASERUN_CS [numeric] |
|
136 distinct values | 2153 (72.1%) | 832 (27.9%) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 47 | TEAM_BATTING_HBP [numeric] |
|
109 distinct values | 1827 (61.2%) | 1158 (38.8%) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 48 | TEAM_PITCHING_H [numeric] |
|
625 distinct values | 2985 (100.0%) | 0 (0.0%) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 49 | TEAM_PITCHING_HR [numeric] |
|
250 distinct values | 2985 (100.0%) | 0 (0.0%) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 50 | TEAM_PITCHING_BB [numeric] |
|
545 distinct values | 2985 (100.0%) | 0 (0.0%) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 51 | TEAM_PITCHING_SO [numeric] |
|
1103 distinct values | 2985 (100.0%) | 0 (0.0%) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 52 | TEAM_FIELDING_E [numeric] |
|
596 distinct values | 2985 (100.0%) | 0 (0.0%) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 53 | TEAM_FIELDING_DP [numeric] |
|
163 distinct values | 2985 (100.0%) | 0 (0.0%) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 54 | pythPercent [numeric] |
|
2922 distinct values | 2985 (100.0%) | 0 (0.0%) | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 55 | era_cat [character] |
|
|
2985 (100.0%) | 0 (0.0%) |
Generated by summarytools 1.0.1 (R version 4.2.2)
2023-02-25
When we group the statistics by the three categories presented earliers, we see a much cleaner density plot across all variables. There are few signs of bimodal distributions, and the skewness of individual variables is reduced greatly.
The training data set exhibits the following characteristics.
| No | Variable | Stats / Values | Freqs (% of Valid) | Graph | Valid | Missing | |||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | yearID [numeric] |
|
151 distinct values | 2389 (100.0%) | 0 (0.0%) | ||||||||||||||||
| 2 | era_cat [character] |
|
|
2389 (100.0%) | 0 (0.0%) | ||||||||||||||||
| 3 | TARGET_WINS [numeric] |
|
106 distinct values | 2389 (100.0%) | 0 (0.0%) | ||||||||||||||||
| 4 | TEAM_BATTING_H [numeric] |
|
572 distinct values | 2389 (100.0%) | 0 (0.0%) | ||||||||||||||||
| 5 | TEAM_BATTING_2B [numeric] |
|
236 distinct values | 2389 (100.0%) | 0 (0.0%) | ||||||||||||||||
| 6 | TEAM_BATTING_3B [numeric] |
|
141 distinct values | 2389 (100.0%) | 0 (0.0%) | ||||||||||||||||
| 7 | TEAM_BATTING_HR [numeric] |
|
256 distinct values | 2389 (100.0%) | 0 (0.0%) | ||||||||||||||||
| 8 | TEAM_BATTING_BB [numeric] |
|
517 distinct values | 2388 (100.0%) | 1 (0.0%) | ||||||||||||||||
| 9 | TEAM_BATTING_SO [numeric] |
|
987 distinct values | 2377 (99.5%) | 12 (0.5%) | ||||||||||||||||
| 10 | TEAM_BASERUN_SB [numeric] |
|
341 distinct values | 2297 (96.1%) | 92 (3.9%) | ||||||||||||||||
| 11 | TEAM_BASERUN_CS [numeric] |
|
126 distinct values | 1715 (71.8%) | 674 (28.2%) | ||||||||||||||||
| 12 | TEAM_BATTING_HBP [numeric] |
|
107 distinct values | 1463 (61.2%) | 926 (38.8%) | ||||||||||||||||
| 13 | TEAM_PITCHING_H [numeric] |
|
587 distinct values | 2389 (100.0%) | 0 (0.0%) | ||||||||||||||||
| 14 | TEAM_PITCHING_HR [numeric] |
|
248 distinct values | 2389 (100.0%) | 0 (0.0%) | ||||||||||||||||
| 15 | TEAM_PITCHING_BB [numeric] |
|
505 distinct values | 2389 (100.0%) | 0 (0.0%) | ||||||||||||||||
| 16 | TEAM_PITCHING_SO [numeric] |
|
1024 distinct values | 2389 (100.0%) | 0 (0.0%) | ||||||||||||||||
| 17 | TEAM_FIELDING_E [numeric] |
|
535 distinct values | 2389 (100.0%) | 0 (0.0%) | ||||||||||||||||
| 18 | TEAM_FIELDING_DP [numeric] |
|
159 distinct values | 2389 (100.0%) | 0 (0.0%) |
Generated by summarytools 1.0.1 (R version 4.2.2)
2023-02-25
DATA PREPARATION
The use of the era_cat variable allows us to group the data set into 3 categories that variables that approach normal distribution.
Since segmenting the data set using the era_cat variable creates 3 categories of variables that approach the normal distribution, we will focus our data preparation step on missing values. The documentation for MICE package recommends that a 5% threshold should be observed for safely imputing missing values. Based on this rule of thumb we will drop the ‘TEAM_BASERUN_CS’ and the ‘TEAM_BATTING_HBP’ variables.
The remaining variables with missing data
(TEAM_BATTING_BB, TEAM_BATTING_SO and
TEAM_BASERUN_SB) will be imputed using the MICE package.
Several methods can be used but for simplicity we selected m=5 the
default method.
Build Model
The first model includes all the variables from the original data set with the exception of the yearID. It could be argued that year might have an impact on the numbers, however we are accounting for year with the era_cat variable. The initial model has an \(AdjR^2 = 0.8078\), and the model selected by stepAIC includes the same variables and has the same \(AdjR^2=8078\).
| Â | TARGET_WINS | |||
|---|---|---|---|---|
| Predictors | Estimates | std. Error | Statistic | p |
| (Intercept) | 81.22 | 3.48 | 23.31 | <0.0001 |
| era_cat1900-1969 | 1.15 | 0.88 | 1.31 | 0.1901 |
| era cat [1969+] | 0.22 | 0.99 | 0.22 | 0.8221 |
| TEAM BATTING H | 0.05 | 0.00 | 26.72 | <0.0001 |
| TEAM BATTING 2B | 0.01 | 0.00 | 2.99 | 0.0028 |
| TEAM BATTING 3B | 0.01 | 0.01 | 1.39 | 0.1643 |
| TEAM BATTING HR | 0.11 | 0.01 | 19.56 | <0.0001 |
| TEAM BATTING BB | 0.04 | 0.00 | 24.09 | <0.0001 |
| TEAM BATTING SO | -0.01 | 0.00 | -8.80 | <0.0001 |
| TEAM BASERUN SB | 0.03 | 0.00 | 10.25 | <0.0001 |
| TEAM PITCHING H | -0.06 | 0.00 | -38.59 | <0.0001 |
| TEAM PITCHING HR | -0.09 | 0.01 | -14.38 | <0.0001 |
| TEAM PITCHING BB | -0.06 | 0.00 | -31.95 | <0.0001 |
| TEAM PITCHING SO | 0.01 | 0.00 | 6.70 | <0.0001 |
| TEAM FIELDING E | -0.01 | 0.00 | -3.40 | 0.0007 |
| TEAM FIELDING DP | 0.05 | 0.01 | 6.76 | <0.0001 |
| Observations | 2389 | |||
| R2 / R2 adjusted | 0.812 / 0.811 | |||
| AIC | 15918.773 | |||
| Â | TARGET_WINS | |||
|---|---|---|---|---|
| Predictors | Estimates | std. Error | Statistic | p |
| (Intercept) | 80.91 | 3.48 | 23.27 | <0.0001 |
| era_cat1900-1969 | 1.06 | 0.87 | 1.21 | 0.2247 |
| era cat [1969+] | -0.10 | 0.96 | -0.10 | 0.9214 |
| TEAM BATTING H | 0.05 | 0.00 | 29.24 | <0.0001 |
| TEAM BATTING 2B | 0.01 | 0.00 | 2.95 | 0.0032 |
| TEAM BATTING HR | 0.11 | 0.01 | 19.57 | <0.0001 |
| TEAM BATTING BB | 0.04 | 0.00 | 24.08 | <0.0001 |
| TEAM BATTING SO | -0.01 | 0.00 | -8.70 | <0.0001 |
| TEAM BASERUN SB | 0.03 | 0.00 | 11.11 | <0.0001 |
| TEAM PITCHING H | -0.06 | 0.00 | -38.63 | <0.0001 |
| TEAM PITCHING HR | -0.09 | 0.01 | -14.67 | <0.0001 |
| TEAM PITCHING BB | -0.06 | 0.00 | -31.93 | <0.0001 |
| TEAM PITCHING SO | 0.01 | 0.00 | 6.58 | <0.0001 |
| TEAM FIELDING E | -0.01 | 0.00 | -3.53 | 0.0004 |
| TEAM FIELDING DP | 0.05 | 0.01 | 6.68 | <0.0001 |
| Observations | 2389 | |||
| R2 / R2 adjusted | 0.812 / 0.811 | |||
| AIC | 15918.721 | |||
The TEAM_BATTING_3B variable was dropped due to low
p-values, giving us the final model with an \(AdjR^2=0.8078\).
| Â | TARGET_WINS | |||
|---|---|---|---|---|
| Predictors | Estimates | std. Error | Statistic | p |
| (Intercept) | 80.91 | 3.48 | 23.27 | <0.0001 |
| era_cat1900-1969 | 1.06 | 0.87 | 1.21 | 0.2247 |
| era cat [1969+] | -0.10 | 0.96 | -0.10 | 0.9214 |
| TEAM BATTING H | 0.05 | 0.00 | 29.24 | <0.0001 |
| TEAM BATTING 2B | 0.01 | 0.00 | 2.95 | 0.0032 |
| TEAM BATTING HR | 0.11 | 0.01 | 19.57 | <0.0001 |
| TEAM BATTING BB | 0.04 | 0.00 | 24.08 | <0.0001 |
| TEAM BATTING SO | -0.01 | 0.00 | -8.70 | <0.0001 |
| TEAM BASERUN SB | 0.03 | 0.00 | 11.11 | <0.0001 |
| TEAM PITCHING H | -0.06 | 0.00 | -38.63 | <0.0001 |
| TEAM PITCHING HR | -0.09 | 0.01 | -14.67 | <0.0001 |
| TEAM PITCHING BB | -0.06 | 0.00 | -31.93 | <0.0001 |
| TEAM PITCHING SO | 0.01 | 0.00 | 6.58 | <0.0001 |
| TEAM FIELDING E | -0.01 | 0.00 | -3.53 | 0.0004 |
| TEAM FIELDING DP | 0.05 | 0.01 | 6.68 | <0.0001 |
| Observations | 2389 | |||
| R2 / R2 adjusted | 0.812 / 0.811 | |||
| AIC | 15918.721 | |||
There are a number of issues with the model diagnostics. The Linearity, and QQ plots show deviation from the straight line at the boundaries. In addition, the reference line for the Homogeneity of Variance graph is not flat. This indicates that residual variance is not consistent across the model values. Further exploration could be conducted to see if there are smaller date ranges and prediction ranges that generate a model that better aligns with the linear regression assumptions.
PREDICT
The next step is to use the selected model to predict the testing data set. The Yardstick package was used to calculate model performance, including the \(R^2\). With an \(R^2=0.8318\) the performance on the testing data set is consistent with the model summary. In the Residual vs. TARGET_WINS graph, there appears to be wins range between 60 and 110 that generates more accurate forecasts. However, it should be noted that the residuals appear to drift downwards. TARGET_WINS less than the mean are overestimated and TARGET_WINS greater than the mean is over estimated.
| .metric | .estimator | .estimate | |
|---|---|---|---|
| 1 | mape | standard | 7.0714 |
| 2 | smape | standard | 6.9789 |
| 3 | mase | standard | 0.3186 |
| 4 | mpe | standard | -0.4950 |
| 5 | rmse | standard | 6.6625 |
| 6 | rsq | standard | 0.7980 |
Bill James developed the Pythagorean winning percentage. The concept strives to calculate the number of games a team should win based on the total offense and the number of runs allowed. Since this model includes total runs scored vs. total runs allowed the expectation is that this model will be a good predictor of a team’s wins in a given season.
\(Win Percentage = (Runs Scored)^2 / [ (Runs Scored)^2 + (Runs Allowed)^2]\)
| Â | TARGET_WINS | |||
|---|---|---|---|---|
| Predictors | Estimates | std. Error | Statistic | p |
| (Intercept) | 4.80 | 0.50 | 9.62 | <0.0001 |
| pythPercent | 152.34 | 0.98 | 154.88 | <0.0001 |
| Observations | 2389 | |||
| R2 / R2 adjusted | 0.909 / 0.909 | |||
| AIC | 14085.472 | |||
As we expected, the Pythagorean model performs well \(Adj R^2\) = 0.9112. However, this linear model differs slightly from the formal definition since it includes an intercept of 5.21 and a coefficient for the Pythagorean factor of 151.39. The formal definition of the model would have an intercept of 0 and a coefficient of 162.
The diagnostic plots show residuals that are normally distributed with a linear relationship to the fitted values. The homogeneity of variance is curved, indicating that the variance of residuals is not consistent.
The next step is to use the Pythagorean Model to predict the testing data set. The Yardstick package was used to calculate model performance, including the \(R^2\). With an \(R^2=0.9131\) the performance on the testing data set is consistent with the model summary. In the Residual vs. TARGET_WINS graph there appears to be a wins range between 50 and 130 that generates more accurate forecasts.
| .metric | .estimator | .estimate | |
|---|---|---|---|
| 1 | mape | standard | 4.9670 |
| 2 | smape | standard | 4.6989 |
| 3 | mase | standard | 0.2031 |
| 4 | mpe | standard | -1.4262 |
| 5 | rmse | standard | 4.4704 |
| 6 | rsq | standard | 0.9187 |
Unsurprisingly, the Pythagorean Model performs well when it comes to win projections. There is clear relationship between the total yearly run differential and the number of games won. It is a simple model but efficient.